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DETAILED ACTION 

1. This is the First Action on the Merits for US Patent Application 10/802,428. 
Currently, claims 1-30 are pending. 

Requirement for Information - 37 CFR 1.105 

2. Applicant and the assignee of this application are required under 37 CFR 1.105 
to provide the information that the examiner has determined is reasonably necessary to 
the examination of this application. 

3. The search of this application reveals that applicants have more complete access 
to information necessary to facilitate understanding of the invention and its context 
relevant to patentability than may be found in typical search techniques. For example, 
Applicant states in paragraph [0013] of the specification that "Regression clustering has 
been studied under a number of different names" and refers to "previous work on RC". 
The paper "Regression Clustering" by Applicant B. Zhang describes a series of papers 
describing other regression clustering techniques, but suggests that the term 
"Regression Clustering" itself used throughout the present application is novel. For 
example, the cited paper "Trajectory Clustering with Mixtures of Regression Models" 
(Gaffney et al.) discloses performing the EM clustering algorithm to sort the general 
motion of a video. Since it appears that what the Inventor claims is patentable is the 
application of the "K-Harmonic Means function" in a regression clustering technique for 
performing image segmentation and subsequent motion detection, the examiner 
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therefore respectfully requires under 37 CFR 105(a)(1)(iii), (vi), and (viii) information 
such as published papers or articles known to the Applicant of the use of other 
clustering algorithms such as linear regression, K-Means or expectation- 
maximization for "estimating motion trials". See MPEP 704.10. 

4. In responding to those requirements that require copies of documents, where the 
document is a bound text or a single article over 50 pages, the requirement may be met 
by providing copies of those pages that provide the particular subject matter indicated in 
the requirement, or where such subject matter is not indicated, the subject matter found 
in applicant's disclosure. 

5. The fee and certification requirements of 37 CFR 1.97 are waived for those 
documents submitted in reply to this requirement. This waiver extends only to those 
documents within the scope of this requirement under 37 CFR 1.105 that are included in 
the applicant's first complete communication responding to this requirement. Any 
supplemental replies subsequent to the first communication responding to this 
requirement and any information disclosures beyond the scope of this requirement 
under 37 CFR 1.105 are subject to the fee and certification requirements of 37 CFR 
1.97. 

6. The applicant is reminded that the reply to this requirement must be made with 
candor and good faith under 37 CFR 1.56. Where the applicant does not have or 
cannot readily obtain an item of required information, a statement that the item is 
unknown or cannot be readily obtained may be accepted as a complete reply to the 
requirement for that item. 
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7. This requirement is part of a full Office action. A complete reply to the enclosed 
Office action must include a complete reply to this requirement. The time period for 
reply to this requirement coincides with the time period for reply to the remainder of the 
Office action. 

Drawings 

8. The drawings are objected to as failing to comply with 37 CFR 1.84(p)(4) 
because reference characters "112" and "120" have both been used to designate a 
vertical pixel location, with "112" used in paragraph [0030] and "120" used in figure 1. 
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to 
4he Office action to avoid abandonment of the application. Any amended replacement- 
drawing sheet should include all of the figures appearing on the immediate prior version 
of the sheet, even if only one figure is being amended. Each drawing sheet submitted 
after the filing date of an application must be labeled in the top margin as either 
"Replacement Sheet" or "New Sheet" pursuant to 37 CFR 1.121(d). If the examiner 
does not accept the changes, the applicant will be notified and informed of any required 
corrective action in the next Office action. The objection to the drawings will not be held 
in abeyance. 

Claim Rejections - 35 USC § 101 

9. 35 U.S.C. 101 reads as follows: 



Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 
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10. Claims 1-12 are rejected under 35 U.S.C. 101 because the claimed invention is 
directed to non-statutory subject matter. In a computer-related claim, the word 
"comprising" is insufficient to describe the relationship between a computer-readable 
medium and a computer program in statutory form. See MPEP §2106.01. It is 
suggested that the word "comprising" be replaced with a concrete term such as 
"encoded with", "storing", "embodied with", "having stored", or "having encoded". 

Claim Rejections - 35 USC § 103 

11. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

12. Claims 1-7, 10-12, 13-19, and 22-28 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over "Motion-based Segmentation Using a Thresholded Merging 
Strategy on Watershed Segments" (De Smet et al.) in view of "K-Harmonic Means - A 
Data Clustering Algorithm" (Zhang et al.), the latter of which is cited in the Information 
Disclosure Statement of 10 September 2004. De Smet et al. discloses using an 
iterative segment-merging technique to determine motion information for an image 
(abstract). Regarding claims 1,13, and 25, in de Smet et al., an initial motion field is 
first determined with a block-matching technique on 4 x 4 blocks (§ 2.1). These initial 
block motion vectors are used for the initial segmentation (§ 2.3). Then, the step of 
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performing the block-based motion estimation for use in the first segmentation 
corresponds with the step of "providing data points". Next, the segments are iteratively 
merged according to similar or shared motion, according to the K-means clustering 
algorithm (§ 2.3). This corresponds with the step of "clustering the data points". When 
this process is finished, the result is a series of large segments corresponding to distinct 
moving regions of an image, each with an associated motion vector (§ 2.3). This 
corresponds with the step of "providing motion estimation". 

De Smet et al. discloses the claimed invention except for performing a K- 
Harmonic Means function to perform regression clustering. Zhang et al. teaches that it 
was known to perform data clustering with the K-Harmonic Means function. Therefore, 
it would have been obvious to one having ordinary skill in the art at the time the 
invention was made to perform clustering based on a K-Harmonic means function, as 
taught by Zhang et al., rather than a linear function such as the K-means function of De 
Smet et al., since Zhang et al. states in the abstract that K-harmonic clustering is less 
sensitive to detrimental effects from sub-optimal initialization that conventional 
clustering techniques. 

Regarding claims 2, 3, 14, and 15, as previously mentioned, de Smet et al. 
produces a motion vector for each segment in an image (§ 2.3). As a result, the most 
important moving areas are determined (§ 3). Regarding claims 10 and 22, in de Smet 
et al., pixels are set as (x,y,t) triples, with x and y as spatial coordinates and t as a time 
coordinate (§ 2.2). Regarding claims 11 and 23, de Smet et al. illustrates motion fields 
(figures 3-6). Although these motion fields are not shown as overlaid on the images, the 
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examiner takes Official Notice that it was well-known in the art at the time of the 
invention to displaying a motion field superimposed on an image to provide a visual 
representation of motion vectors. See for example the demonstration by R. Carceroni 
available online, cited in Form 892. Regarding claims 12 and 24, de Smet et al. 
illustrates highlighted motion segments overlaid on an image (figures 11 and 12). 

Regarding claims 4, 16, and 26, Zhang et al. teaches selecting K centers m(l) 
from N data points x(i) (pg. 1), initializing regression functions by setting the partial 
derivatives of a performance function with respect to each center position to zero (pg. 

4) , calculating distance d(i,l) between data point x(i) and center point m(l) (pg. 4), 
calculating membership probability q(i,k) (pg. 5), recursively calculating new m(k), (pg. 

5) , and stopping when the recursively-calculated performance value stabilizes, that is, 
when its change with each iteration becomes small (pg. 5). Regarding claims 5 and 17, 
a clustering in which initialization is randomized is described (pg. 11). Regarding claims 
6 and 18, KHM is described as a recursive function that performs until stabilization is 
achieved (pg. 5). Regarding claims 7 and 19, insensitivity to initialization is an inherent 
result of the K-Harmonic means algorithm (abstract). 

13. Claims 8, 9, 20, and 21 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over de Smet et al. in view of Zhang et al. as applied to claims 4 and 16 
above, and further in view of "A Video Segmentation Algorithm for Hierarchical Object 
Representations and its Implementation" (Herrmann et al.). Claims 8, 9, 20, and 21 
disclose using color information to segment images, but de Smet et al. only discloses 
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"standard watershed techniques" (§ 2.2) to perform the initial segmentation without 
providing details. 

Herrmann et al. discloses a method for image segmentation to extract objects 
from a moving image. Regarding claims 8, 9, 20, and 21, after an initial block-matching 
motion estimation, similar to de Smet et al., images are segmented according to specific 
color information, followed by shape analysis, and lastly motion analysis to merge 
regions to determine objects (§ II). This corresponds with the claimed "predetermined 
criteria". In color analysis, a region is determined as homogeneous if the pixel 
difference in the region is below a threshold. Homogenous, connected areas are 
determined as "quasi-flat zones". These quasi-flat zones are further processed and 
become the basis for further segmentation (§ II. B). This corresponds with "portioning 
data according to color". 

De Smet et al., in combination with Zhang et al., disclose a majority of the 
features of claims 8, 9, 20, and 21 as discussed above, the claimed invention except for 
color segmentation. Herrmann et al. teaches that it was known to segment a moving 
image according to color. Therefore, it would have been obvious to one having ordinary 
skill in the art at the time the invention was made to determine image segmentation by 
color as taught by Herrmann et al., since Herrmann et al. teaches in page 205, third 
paragraph, that color analysis produces the most accurate type of segmentation. 

14. Claims 27-30 are rejected under 35 U.S.C. 103(a) as being unpatentable over de 
Smet et al. in view of Zhang et al., and in view of US Patent 6,084,912 A (Reitmeier et 
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al.) Claims 27-30 are in means-plus-function format, complying with 35 U.S.C. 112, 
sixth paragraph. Accordingly, the "system" of claims 27-30 will be limited to a general- 
purpose computer such as a PC, as illustrated in figure 5 of the specification of the 
present invention. Although it is implied that the algorithms of de Smet et al. and Zhang 
et al. are computer-operated, neither de Smet et al. nor Zhang et al. explicitly teach this. 

Reitmeier et al. discloses a video encoder. This encoder may operate on 
MPEG-4 video (column 1: line 57), as specified in paragraphs [0006] and [0007] of the 
present invention as a codec on which the present invention is applied. Regarding 
claims 27 and 28, the encoder of Reitmeier et al. may operate as a software application 
on a general-purpose computer (column 2: lines 64-67). Regarding claims 29 and 30, 
Reitmeier et al. discloses frame memory 155, which stores decoded reference frames 
for motion compensation (column 7: line 27), as is well-known in MPEG encoders. 

De Smet et al., combined with Zhang et al., discloses the claimed invention 
except for encoding video on a general-purpose computer. Reitmeier et al. teaches that 
it was known to implement an MPEG-4 encoder as software embedded on a computer. 
Therefore, it would have been obvious to one having ordinary skill in the art at the time 
the invention was made to encode the system of de Smet et al. and Zhang et al. in a 
software MPEG encoder embedded on a computer, as taught by Reitmeier et al., in 
order to perform computationally complex functions such as motion compensation, 
quantization, and variable-length encoding inherent in the video coding process. 
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Conclusion 

15. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. US Patent 4,253,065 A (Richards et al.) teaches a system that 
overlays video frames to produce a motion blur effect. US Patent 5,734,737 A (Chang 
et al.) teaches a method for segmenting an image according to moving objects. US 
Patent 5,995,668 A (Corset et al.) teaches coding a segmented image. US Patent 
6,160,846 A (Chiang et al.) teaches performing a regression model to determine a 
quantization level in video coding. US Patent 6,442,202 B1 (Borer) teaches a method 
for determining motion vector error based on regression methods. US Patent 6,473,462 
B1 (Chevance et al.) and French Patent Application Publication 2,833,797 A1 (le Clerc 
et al.) teaches determining the dominant motion of an image based on a regression 
algorithm. "Trajectory Clustering with Mixtures of Regression Models" (Gaffney et al.) 
teaches clustering videos into having various types of movements based on the EM 
algorithm. "An Iterative Motion Estimation - Segmentation method Using Watershed 
Segments" (Patras et al.) teaches segmentation of an image according to moving 
objects. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to David N. Werner whose telephone number is (571) 272- 
9662. The examiner can normally be reached on Monday-Friday from 8:30 AM - 5:00 
PM. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mehrdad Dastouri, can be reached on (571) 272-7418. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic i 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



DNW 




